# Grouped query attention

Mistral-Nemo-Base-2407 is a 12-billion-parameter generative text pre-training model jointly trained by Mistral AI and NVIDIA, outperforming models of similar or smaller scale.

Meta Llama 3.1 is a large language model series supporting 8 languages, available in 8B/70B/405B scales, outperforming most open-source and proprietary chat models in industry benchmarks

Mistral-7B-Instruct-v0.1 is an instruction fine-tuned version based on Mistral-7B-v0.1, suitable for dialogue generation tasks.